NVIDIA Sets New Benchmarks for AI Inference Economics with Vera Rubin and GB300 NVL72 Performance Breakthroughs

The landscape of artificial intelligence is currently defined by a relentless drive for efficiency, where the economic viability of large-scale model deployment hinges on three critical pillars: raw system performance, the ability to scale infrastructure seamlessly, and the velocity of continuous software optimization. As organizations transition from the experimental phase of AI to industrial-scale production, the demand for hardware that can deliver higher token throughput while reducing the cost-per-inference has reached an all-time high. Today’s release of the MLPerf Inference v6.1 results provides a definitive look at how the latest NVIDIA architectures—specifically the Vera Rubin NVL72 and the Grace Blackwell GB300 NVL72—are setting new standards for these economic levers.

The Evolution of Inference Economics

At the core of the modern AI data center is the concept of platform fungibility. Unlike specialized hardware of the past, contemporary AI infrastructure must be versatile enough to run diverse workloads—ranging from massive language models and complex video generation to real-time agentic reasoning—without requiring specialized, isolated silos. By maintaining high utilization across all these workloads, data center operators can maximize their return on capital expenditure.

NVIDIA’s strategy, as evidenced by the v6.1 submission data, focuses on full-stack co-design. By engineering hardware and software in tandem, the company aims to eliminate bottlenecks at every stage of the inference pipeline. This approach is particularly vital as AI models shift from simple text generation to agentic systems that require multi-step reasoning, planning, and execution—tasks that impose significantly higher demands on memory bandwidth and interconnect latency.

Chronology of the MLPerf v6.1 Submissions

The MLPerf benchmark suite has become the industry-standard yardstick for measuring AI performance in real-world scenarios. The v6.1 results, published on September 16, 2026, represent a significant milestone in the maturation of Blackwell-era hardware.

The timeline leading up to this release saw a rapid iterative process. Following the v6.0 results, NVIDIA focused heavily on refining the interaction between its TensorRT-LLM library and the underlying hardware. These efforts resulted in marked performance gains in a matter of months. Notably, the preview results for the Vera Rubin NVL72—the latest architecture in the NVIDIA roadmap—were submitted alongside mature GB300 deployments, providing a clear trajectory for performance scaling as the platform transitions toward broader availability.

Performance Data and Hardware Benchmarks

The data submitted to MLCommons illustrates a substantial leap in capabilities. On the DeepSeek-R1 benchmark, the Vera Rubin NVL72 architecture demonstrated up to 2.5x higher throughput compared to the GB300 NVL72. Even more striking is the performance on the Qwen3-VL model, where the Vera Rubin platform achieved up to 3.7x higher throughput across offline, server, and interactive scenarios.

These gains are not merely the result of brute-force transistor counts. They are driven by specific hardware-level innovations:

• Enhanced Tensor Cores and Transformer Engine: These components have been optimized to accelerate both the prefill and decode stages, which are historically the most time-consuming parts of the inference process.
• NVFP4 Precision: By reducing the memory footprint of model weights and the KV cache, NVIDIA has effectively increased the number of tokens that can be processed concurrently without sacrificing output quality.
• Disaggregated Serving: By separating the prefill and decode operations, the system can allocate resources more effectively, particularly for "Mixture-of-Experts" (MoE) models like DeepSeek-R1.

The interconnect foundation is equally critical. The sixth-generation NVLink and NVLink Switch provide a scale-up domain that delivers 10x higher packet rates and 3x lower latency than standard Ethernet-based networking. This allows the system to function as a single, massive GPU, ensuring that the performance gains are not lost to communication overhead at the rack level.

Scaling Efficiency: The Rack-Scale Advantage

A common trap in scaling AI infrastructure is the "diminishing returns" phenomenon, where adding more GPUs results in less than proportional increases in throughput. NVIDIA’s GB300 NVL72 submissions demonstrate that this can be avoided with the right architectural approach.

In testing involving the DeepSeek-R1 model, NVIDIA scaled from a single rack of 72 GPUs to a four-rack cluster of 288 GPUs. The result was a 99% scaling efficiency in the offline scenario, meaning that the throughput increased almost linearly with the hardware added. This level of efficiency is paramount for cloud service providers and hyperscalers, as it ensures that capital investments translate directly into increased capacity rather than being squandered on administrative or communication bottlenecks.

Furthermore, on the WAN 2.2 text-to-video benchmark, the GB300 NVL72 reached 0.65 videos per second at a resolution of 720p. This represents a 9x throughput increase and a 7.5x reduction in latency compared to single-node deployments, highlighting the efficiency gains of rack-scale orchestration.

Software as a Performance Multiplier

The MLPerf results highlight the impact of NVIDIA’s "software velocity." Between v6.0 and v6.1, the GB300 NVL72 saw a 1.6x improvement in Qwen3-VL performance. These gains were achieved entirely through software: lower KV cache precision, enhanced kernel fusion, and more efficient request orchestration using the NVIDIA Dynamo and vLLM frameworks.

This continuous optimization cycle is now a standard feature of the platform. Post-submission internal testing on GPT-OSS-120B and DLRMv3 models suggests that even greater efficiencies are being unlocked, which will likely be reflected in future iterations of the platform software. This implies that the total cost of ownership for an NVIDIA-based system continues to improve over the life of the hardware, effectively providing "free" performance upgrades through firmware and library updates.

Agentic AI and Future Benchmarking

The emergence of agentic AI—systems capable of autonomous reasoning—has created a new paradigm for performance measurement. Traditional throughput metrics, while useful, fail to capture the nuances of multi-step agentic workflows. In response, tests like the SemiAnalysis AgentX benchmark have been introduced. In these preview tests, the Vera Rubin NVL72 demonstrated a 30x performance improvement over the GB300 NVL72.

The upcoming MLPerf Endpoints benchmark is expected to standardize these agentic measurements. By moving beyond simple token generation and toward task-completion metrics, the industry is aligning its hardware testing with the actual, real-world utility of modern AI models.

Ecosystem and Broad Industry Impact

The strength of the NVIDIA platform is reflected in its widespread adoption. The v6.1 submission featured 19 partners, including major cloud providers like Azure and Oracle Cloud Infrastructure, as well as specialized AI infrastructure firms like CoreWeave and Nebius.

The inclusion of companies like ASUS, Cisco, Dell Technologies, HPE, and Supermicro demonstrates that the hardware is ready for deployment across diverse environments—from high-performance computing centers to corporate data centers. The fact that eight of these partners submitted results on multi-node Blackwell NVL72 systems confirms that the architecture is not merely a lab curiosity, but a mature, production-ready solution.

Conclusion and Economic Implications

For organizations tasked with designing the AI factories of the future, the choice of infrastructure is a high-stakes decision. The data from MLPerf Inference v6.1 indicates that the gap between leading architectures is widening. By focusing on rack-scale interconnects, software-driven optimization, and specialized hardware for agentic reasoning, NVIDIA has established a framework where inference costs can be managed even as model complexity grows.

The economic implications are clear: by maximizing token generation per watt and per dollar, providers can lower the barrier to entry for complex AI applications. As the industry moves toward the widespread adoption of autonomous agents, the ability to maintain high throughput and low latency will remain the ultimate competitive advantage. With the Vera Rubin platform, NVIDIA appears to be positioning itself to maintain this trajectory, providing the foundational technology necessary for the next phase of the AI revolution.